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Importance: Governments have introduced non-pharmaceutical interventions (NPIs) in 
response to the pandemic outbreak of Coronavirus disease (COVID-19). While NPIs aim 
at preventing fatalities related to COVID-19, the previous literature on their efficacy has 
focused on infections and on data of the first half of 2020. Still, findings of early NPI 
studies may be subject to underreporting and missing timeliness of reporting of cases. 
Moreover, the low variation in treatment timing during the first wave makes identification 
of robust treatment effects difficult. 


Objective: We enhance the literature on the effectiveness of NPIs with respect to the 
period, the number of countries, and the analytical approach. 


Design, Setting, and Participants: To circumvent problems of reporting and treatment 
variation, we analyse data on daily confirmed COVID-19-related deaths per capita from 
Our World in Data, and on 10 different NPls from the Oxford COVID-19 Government 
Response Tracker (OxCGRT) for 169 countries from 1st July 2020 to 1st September 
2021. To identify the causal effects of introducing NPIs on COVID-19-related fatalities, 
we apply the generalized synthetic control (GSC) method to each NPI, while controlling 
for the remaining NPls, weather conditions, vaccinations, and NPI-residualized COVID-19 
cases. This mitigates the influence of selection into treatment and allows to model flexible 
post-treatment trajectories. 


Results: We do not find substantial and consistent COVID-19-related fatality-reducing 
effects of any NPI under investigation. We see a tentative change in the trend of 
COVID-19-related deaths around 30 days after strict stay-at-home rules and to a slighter 
extent after workplace closings have been implemented. As a proof of concept, our 
model is able to identify a fatality-reducing effect of COVID-19 vaccinations. Furthermore, 
our results are robust with respect to various crucial sensitivity checks. 

Conclusion: Our results demonstrate that many implemented NPIs may not have 
exerted a significant COVID-19-related fatality-reducing effect. However, NPIs might have 
contributed to mitigate COVID-19-related fatalities by preventing exponential growth in 
deaths. Moreover, vaccinations were effective in reducing COVID-19-related deaths. 
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HIGHLIGHTS 


- We enhance the existing literature on the effect of non- 
pharmaceutical interventions (NPIs) on COVID-19-related 
deaths with respect to the period (1), the number of countries 
(2), and the analytical approach (3). 

- (1) We analyze data on COVID-19-related deaths per capita 
from Our World in Data, and on 10 different NPIs from 
the Oxford COVID-19 Government Response Tracker of 
169 countries from 1st July 2020 to 1st September 2021 
to attenuate identification problems stemming from low 
variation in treatment (timing) and data quality. 

- (2) Compared to other studies, we use data of more countries 
(n = 169) covering 98% of the world population. 

- (3) We apply the Generalized Synthetic Control (GSC) 
method to mitigate the influence of selection into treatment 
and to model flexible post-treatment trajectories. 

- Applying the GSC method, we do not find substantial and 
consistent COVID-19-related fatality-reducing effects of any 
NPI under investigation. 

- Using the same approach, we find however a 
significant and substantial fatality-reducing effect of 
COVID-19 vaccinations. 

- The results of our study provide further guidance 


to judge the effectiveness of NPIs for reducing 
COVID-19-related fatalities. 

INTRODUCTION 

Governments have introduced non-pharmaceutical 


interventions (NPIs) in response to the pandemic outbreak 
of Coronavirus disease (COVID-19) since early 2020 (see 
Supplementary Figures 1, 2). NPIs include traditional 
epidemiological instruments like public information campaigns, 
testing strategies, as well as contact tracing, and the isolation 
of infected and vulnerable people. Moreover, obligations to 
wear a face mask have been introduced. A large share of 
countries has implemented various lockdown-related NPIs. 
These include the closure of schools, workplaces and public 
transport, restrictions on public events and gatherings, and 
more stringent measures like stay-at-home requirements, 
as well as restrictions on domestic and international 
movement (1). 

While the potential benefits of stringent NPIs are 
straightforward (mitigate COVID-19-related fatalities and 
maltreatment of other diseases due to health system overload), 
they are likely to be jeopardized by potentially severe or even 
“prohibitive” (2) economic, social, and public health-related 
negative externalities associated with the implementation of 
these NPIs (3-8). Hence, evaluation of the proportionality of 
NPIs heavily hinges on consistent empirical evidence of their 
efficacy in the ongoing pandemic (9). 

Ever since, a valuable body of empirical NPI-studies has 
emerged to judge the efficacy of various measures (see 
Supplementary Table 1 in the Supplementary Material Part E 
for an overview). However, several potential limitations of the 
previous literature impede the judgment of the NPIs’ efficacy. 
First, NPIs ultimately aim at avoiding fatalities related to 


COVID-19, e.g., by preventing the health system to collapse. 
However, earlier studies on their efficacy have primarily 
focused on infections. Only ten of the 37 studies listed in 
Supplementary Table 1 investigate the effect of NPIs on COVID- 
19-related deaths (10-19). Although studies on infections 
provide important insights on how NPIs affect the disease 
dynamics, they cannot adequately evaluate the main aim of 
avoiding fatalities. Furthermore, underreporting of infections 
may be more pronounced than for fatalities. Hence, it is favorable 
to use data on fatalities (20). Second, former investigations based 
on data of the first half of 2020 may be subject to underreporting 
and missing timeliness of reporting during the first viral outbreak 
(14, 17). Third, the identification of robust treatment effects 
of NPIs is difficult based on the first wave because of the low 
variation in treatment type and treatment timing across countries 
in early 2020 (17, 21) (see Supplementary Figures 3-15). 

Against this backdrop and given the heterogeneity in 
data used, methods applied, and NPIs, time spans and 
countries/territories analyzed, the results of early NPI studies 
vary considerably. Early causal analyses on daily COVID-19- 
related fatalities accounting for population size and the temporal 
delay of treatment impacts find substantial mitigating effects 
for the initial “lockdown” (12, 14), school closure (10, 13, 
18, 19), workplace closure (18, 19), cancellation of public 
events (18), stay-at-home orders (16), travel restrictions (15), 
and mask obligations (11, 15). However, all these studies 
are based on data of the first wave. Only one study (17) 
investigates fatalities after the first half of 2020 (August 2020 
to January 2021). Based on a hierarchical Bayesian transmission 
model applied to 7 countries/114 subnational entities, the study 
(17) identifies substantial mitigating effects for school closure, 
workplace closure, and firm restrictions on gatherings (see 
Supplementary Table 1). 

We enhance the existing literature on the effect of NPIs 
on COVID-19-related deaths with respect to the period, 
the number of countries, and the analytical method. First, 
we analyse data starting from the second half of 2020 (lst 
July 2020 to Ist September 2021) to circumvent problems 
of reporting and treatment variation. Variation in treatment 
timing is sufficiently high between countries in this time 
span (Supplementary Figures 3-12). Second, compared 
to other studies (17), we use data of more countries 
(N = 169) covering 98% of the world population. Third, 
to estimate the average treatment effect on the treated 
(ATT), we apply the Generalized Synthetic Control (GSC) 
method (22). The GSC approach is a combination of the 
synthetic control approach (23-25) and traditional difference- 
in-differences (DiD) methods (26). This attenuates the 
influence of selection into treatment while modeling flexible 
post-treatment trajectories. 


MATERIALS AND METHODS 
Data 


We combine different data sources on the country level 
(N = 169 countries). First, we derive daily information on 
confirmed COVID-19 deaths per capita from Our World in 
Data (27). Second, we add daily information on various NPIs 
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and governmental stringency from the Oxford COVID-19 
Government Response Tracker (OxCGRT) (1). Our main 
outcome measure is the daily number of newly confirmed deaths 
attributed to COVID-19 per million inhabitants. We coded all 
days as missing on which a country reported a negative number 
of deaths, which happened in a few instances when countries 
corrected earlier numbers. 

From OxCGRT (1) we use the following 10 NPIs: school 
closing, workplace closing, closure of public transport, stay at 
home rules, restriction of internal movement, restrictions on 
international travel, protection of the elderly, testing policy, 
contact tracing, and mask obligations. OxCGRT codes a country’s 
stringency in each of these domains with different categories. 
For our main analyses we recoded all NPIs as binary indicators, 
taking the value one (implemented) if a country is in the highest 
available category of each measure on a given day, and zero 
otherwise. Supplementary Figures 3-12 depict the stringency 
sequences for each NPI. 


Analytical Strategy 

Unobserved differences between countries and temporal 
shocks likely influence the number of COVID-19 related cases 
and deaths. Hence, a natural choice would be a difference- 
in-differences (DiD) like estimator (28), solely relying on 
within country and period variances for identification 
(28, 29). However, conventional within-estimators face 
several methodological challenges when aiming at identifying 
the impact of NPIs on COVID-19 related fatalities, such 
as selection on pre-treatment trends or treatment effect 
heterogeneity (26, 30-32). 

We tackle this problem by applying a Generalized Synthetic 
Control (GSC) method (22). The intuitive idea behind the 
conventional Synthetic Control method is to construct a 
synthetic control unit for a single treated unit by re-weighting 
observations from the pool of control-units (23-25). Among 
other characteristics, the re-weighting is based on the pre- 
treatment pathway of the outcome. The method thus compares 
the treatment unit to a weighted control-pool which, on average, 
has a similar pre-treatment outcome trajectory. Moreover, 
by predicting the counterfactual outcome for the treated 
observations, heterogeneous treatment effects over time are 
flexibly identified. 

The traditional Synthetic Control approach applies only to 
the case of one treated unit. Yet, the GSC method (22) provides 
a framework which generalizes the synthetic control method to 
multiple treated units by using factor augmented models. We 
start with the general model 


Yit = ôitDit + Xip + Lit + eit, (1) 


where Yj; is the dependent variable (fatalities per capita), X; 
are time-varying controls, and sgip the idiosyncratic error. Dit 
is a binary treatment indicator, and ô; denotes the treatment 
effect for each unit and time-period, with 5; being the average 
treatment effect on the treated (ATT). Li subsumes unobserved 
factors approximating the outcome trend. The original GSC 
method based on interactive fixed effects models directly 


estimates L = AF, with A being an N x r matrix of unknown 
factor loadings (unit-specific intercepts), and F an r x T matrix 
of unobserved common factors (time-varying coefficients) (22). 
This however requires the number of factors r to be specified 
correctly, for instance by using cross-validation methods. 

In contrast to interactive fixed effects models, the matrix 
completion method (33) does not estimate A and F, but directly 
estimates L based on nuclear norm regularization. Intuitively, 
L is derived by minimizing the sum of squared differences 
between Yj; and Lj (plus non-regularized unit and time fixed 
effects) based on pre-treatment observations while adding a 
penalty term A ||L||—similar to the Least Absolute Shrinkage and 
Selection Operator (LASSO) estimator. The matrix completion 
method in GSC is used to estimate the ATT in the following 
way (22, 34). First, B and L are estimated based on the pool of 
control units, where 4 is determined by cross-validation (33). 
Here, we use 20 possible à values and 10-fold cross-validation. 
Second, the counterfactual outcome for the treated units is 
estimated by Yi(D = 0) = XB + iy. Third, based on the 
predicted counterfactual outcome, we can then estimate the ATT 
5 it = Yi(D = 1) - Ê; (D = 0). For inferential statistics, we provide 
confidence intervals based on non-parametric bootstraps of 1,000 
runs clustered at the country level (see below). For estimation, we 
use the R package gsynth v.1.1.9 (22). 

One limitation of GSC in our case is that the method is 
designed for settings of staggered treatment adoption. Countries 
can, however, transition from treatment (having an NPI 
implemented) to control (relaxing the NPI), and going back to 
treatment (see Supplement B for treatment sequences). Thus, 
we divide our data into country-period splits, where a country 
is treated as a new unit with each transition from treatment to 
control. A single country can, for instance, act as treated unit 
in early periods and as control unit in later periods. This also 
ensures that we do not conflate our temporal treatment effect of 
implementation with relaxations of NPIs. 

To analyse the effectiveness of each single NPI; with i € 
{1, .. 10}, we simultaneously control for the stringency index 
in all other NPIs: Index;, j # i. We follow the OxCGRT 
methodology for normalized sub-index scores by defining 
Index; = 100°“) 
of the measure, N,j the maximum category (highest stringency), 
fj a binary flag indicating whether a measure is geographically 
targeted (0) or general (1), and F; indicates if measure j has a flag 
or not (e.g., international travel restrictions are always general). 

We control for the cumulative number of vaccinations 
per capita (27), as vaccinations likely affect both—the 
implementation of NPIs and the number of fatalities. Because 
COVID-19 transmissions (35-37) as well as COVID-19 
related morbidity (38) might be correlated with seasonal 
weather conditions, we also control for the monthly average 
temperature and temperature squared, cloud cover, specific 
humidity, and precipitation. These data were derived from 
the ERA5 reanalysis data based on Copernicus Climate 
observations (39). 

Moreover, we control for temporal lags of residualized 
COVID-19 cases. Besides the COVID-19 deaths, also the cases 





, where v; is the current category 
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are likely to impose a strong effect on the likelihood of 
implementing NPIs. However, naively controlling for cases 
would lead to an overcontrol-bias, as NPIs reduce deaths not 
only directly but also through a reduction in infections. We thus 
follow a regression-with-residuals approach (40, 41), making 
COVID-19 cases orthogonal to past NPI implementations. 
More specifically, we first regress cases on various time- 
lags of NPIs within countries and time-periods: Cases; = 
ra 2 NPI +a; + & for all K NPIs and L temporal 
lags, where œ; and & are country and time fixed effects. 
Subsequently, we derive the residuals of this first-stage regression. 
This leaves us with the residualized COVID-19 cases which 


are independent of NPI implementations within the past 7-35 
days. In our main analysis, we control for the residualized 7- 
day backwards rolling average of residualized COVID-19 cases 
at five different temporal lags: t - 7, t - 14, t - 21, t - 28, 
and t - 35, including second and third order polynomials. 
The high number of lags in the first and second stage intends 
to relax a-priori assumptions on the temporal dependency 
between COVID-19 cases, deaths, and NPIs. Controlling for 
the residualized number of COVID-19 cases notably reduces 
the pre-treatment differences in control and treatment group 
(see Supplementary Figure 16 for results without controlling for 
residualized cases). 
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FIGURE 1 | The effect of NPIs on COVID-19 deaths. Generalized synthetic control estimator based on daily data. Black solid lines represent the average treatment 
effects on the treated (ATTs). Ribbons represent 95% non-parametric confidence intervals based on 1,000 bootstrap runs. Dotted lines are the null lines. Dashed lines 
represent linear predictions based on the 35 days before the intervention. Controls: 9 remaining NPIs as stringency index, temperature, temperature?, cloud cover, 
precipitation, humidity, total vaccinations, 7-day backwards rolling average of NPI-residualized COVID-19 cases at t- 7, t- 14, t- 21, t- 28, and t - 35. 
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RESULTS 


Relying on the analytical approach above, we observe that none 
of the strictly implemented NPIs under investigation had a 
substantial and consistent effect on COVID-19-attributed deaths 
over time (see Figure 1). None of the post-treatment trajectories 
differs significantly from the null-line. Even when comparing 
the post-treatment trajectories to the linear extrapolation of the 
35-days pre-treatment period (dashed line), only strict stay-at- 
home requirements produce borderline-significant differences. 
We only observe a tentative change in the trend of COVID- 
19-related deaths starting around 30 days after strict stay-at- 
home rules have been introduced, but this does not exert a 
statistically significant effect. Similarly, we observe a tentative 
change in the trend of COVID-19-related fatalities 30 days 
after workplaces have been closed. However, the effects are 
not statistically different from zero. If we take this as evidence 
for a mitigating effect depends on whether we only consider 
reductions in the total of COVID-19-related deaths as mitigation 
or whether we also consider the option of preventing exponential 
growth in COVID-19-related deaths—an assumption we cannot 
test adequately based on the available data (see also Discussion). 

We conducted a number of robustness checks (see 
Supplementary Material C). First, we carried out a proof- 
of-concept analysis to demonstrate that our model is able to 
detect a substantial and statistically significant ATT based on 
the data at hand: For this purpose, we investigated the effect of 
a pharmaceutical intervention (PI) instead of NPIs on fatalities. 
More specifically, we analyzed the impact of vaccinations against 
COVID-19 (27) on COVID-19-related deaths per 1 miocapita. 
Vaccination intervention is coded as treatment if the share of 
disseminated vaccine doses per capita exceeds 80%. Additionally, 
the model controls for all 10 NPIs under investigation (stringency 
index) and the controls listed above. As Figure 2 demonstrates, 
our model identifies a consistent and statistically significant 
mitigating effect of vaccination on COVID-19 fatalities from 
around day 45 to day 110 after treatment. This effect is also 
of substantial magnitude: for a country of 60 mio inhabitants, 
we estimate that vaccinations prevented around 90 death per 
day from 45 days onwards after reaching the threshold of 80 
vaccinations per 100 inhabitants. 

Second, we were concerned that spatial spillovers could affect 
our main results (42). Thus, we additionally controlled for NPI 
implementations in neighboring countries. The results indicate 
that spatial spillovers do not alter the results substantially 
on the national level (see Supplementary Figure 17). Third, 
we coded the highest two categories as intervention. Results 
(Supplementary Figure 18) do not indicate meaningful 
influences on fatalities, with the one exception of school 
closings. Note however that this result is rather unstable, 
as we lack a sufficient number of control units. Fourth, the 
results remain stable when controlling for the number of other 
interventions implemented instead of a stringency index (see 
Supplementary Figure 19). Fifth, we tested if results differ when 
using first-wave data only. Results with data before September 
2020 (Supplementary Figure 20) deviate from our main findings 
only with regard to restrictions on internal movement. With first 
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FIGURE 2 | The effect of vaccinations (vaccine doses per inhabitant >80 %) 
on COVID-19 deaths. Generalized synthetic control estimator based on daily 
data. The black solid line represents the average treatment effect on the 
treated (ATT). Ribbons represent 95% non-parametric confidence intervals 
based on 1,000 bootstrap runs. The dotted line is the null line. The dashed line 
represents the linear prediction based on the 35 days before the intervention. 
Controls: 10 NPIs as stringency index, temperature, temperature*, cloud 
cover, precipitation, humidity, 7-day backwards rolling average of 
NPIl-residualized COVID-19 cases at t- 7, t-14, t- 21, t- 28, and t - 35. 











wave data, we observe a downturn in fatalities after restrictions 
on internal movement. However, the ATT is estimated with large 
insecurity. Sixth, we checked if our results depend on treatment 
timing. It is reasonable to assume that the implementation of 
NPIs in an early phase of the viral outbreak is more effective 
(19, 43). We therefore re-estimated our models separately for 
early and late adopters. We calculated the 7-day backwards 
rolling average of fatalities at treatment intervention and defined 
units below (above) the overall median as early (late) adopters. 
For early adopters (Supplementary Figure 21) the stay-at- 
home effect disappears, while it is stronger for late adopters 
(Supplementary Figure 22). This gives rise to the notion that 
stay-at-home rules have been implemented as a last resort to 
respond to a steep increase in cases. For all other NPIs the results 
are substantially similar. 


DISCUSSION 


Based on the applied GSC approach, we do not find substantial 
COVID-19-related fatality-reducing effects of the ten NPIs under 
investigation. We see a tentative change in the trend of COVID- 
19-related deaths starting around 30 days after stay-at-home rules 
have been introduced, and a slighter turn after workplace closing. 

Our results do not corroborate the findings of former studies 
with less countries relying on first wave data, and following 
different, mostly DiD type analytical approaches on school 
closures (10, 13, 17-19), workplace closures (17-19), stay at home 
orders (16, 18), restrictions on international travel (15), and mask 
obligations (11, 15). These differences may also emerge from 
underreporting and missing timeliness of reporting of fatalities 
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during the first viral outbreak (14, 17). Yet, our results mirror 
the findings of a recent study (32) following a conceptually 
similar analytical strategy to attenuate estimation bias stemming 
from self-selection into treatment while investigating the effect 
of 46 US state-level shelter-in-place orders on COVID-19 cases. 
In contrast to a DiD type approach, the study (32) finds 
no significant impacts of shelter-in-place orders on COVID- 
19 cases. 


LIMITATIONS 


Nonetheless, our estimates might be subject to several issues. 
First, differences in population-wide compliance to NPIs might 
serve as an explanation for our findings (44, 45). This is 
more likely for mask obligations, contact tracing, testing 
policies, protecting elderly, and domestic travel restrictions. For 
instance, the tentative downwards trend after internal movement 
restrictions during the first wave (Supplementary Figure 20) 
may indicate higher compliance during this period compared 
to later periods. However, compliance rates are unlikely to 
serve as substantial explanation for the absence of mitigating 
effects of school and workplace closings, the closure of public 
transport, and restrictions on international travel. Second, our 
statistical approach complicates the investigation of simultaneous 
NPI implementations (46, 47). Comparing selected combinations 
of two simultaneous NPIs to countries without any of the 
two, however we do not find any consistent significant effect 
(Supplementary Figure 23). 

Third, while the applied GSC approach attenuates the 
potential bias of self-selection into NPI implementation (pre- 
treatment), it cannot rule out the possibility of a potential post- 
treatment exponential growth in fatalities on our estimates. In 
other words, we might assume that these countries implementing 
the measures would otherwise have had an exponential growth in 
fatalities, and implementing these measures kept them relatively 
close to pre-treatment levels. Based on the available data, 
however, we cannot test this assumption. 

Hence, our finding of statistically insignificant effects does 
not necessarily contradict the notion that the implementation 
of NPIs might have contributed to mitigate COVID-19-related 
fatalities by preventing exponential growth (rather than a decline 
in total COVID-19-related deaths). 

Furthermore, data quality and local context might influence 
our results: There might still be misreporting of COVID- 
19-related deaths in non-hospitalized fatalities (20). The 
local context associated with the specific implementation 
of NPIs (e.g., school closure) might influence our results, 
as the relatively broad categories of OxCGRTI might not 
capture variation in implementation, e.g., across subnational 
territories (21). 

Nevertheless, we demonstrate that high vaccination rates 
against COVID-19 help to reduce fatalities while controlling for 
the implementation of 10 NPIs. This highlights the importance of 


PIs to combat the ongoing COVID-19 pandemic. However, the 
effectivity of vaccines hinges on continuous collaborative efforts 
to adapt COVID-19 vaccines given the spread of new variants 
(48) and on an increase in vaccination rates (49). The differential 
effectivity of NPIs and PIs might be related to the pronounced 
age gradient in COVID-19 related fatalities and associated multi- 
morbidities (50). This might especially explain why we find 
effects of workplace closing, public transport closing and contact 
tracing on COVID-19 cases (Supplementary Figure 24), but do 
not see significant effects on COVID-19 fatalities. 

Altogether, the present study enhances the literature on the 
effectiveness of NPIs with respect to the period, the number 
of countries, and the analytical approach. However, some 
limitations associated with data quality and availability as well as 
the analytical strategy remain to be addressed by future research. 
Hence, we cannot test if the NPIs might have prevented a 
potential exponential growth in COVID-19 fatalities in those 
countries who selected into treatment. Still, the results of our 
study add new insights on the public health effects of NPIs and 
can thus help to guide future responses to pandemic outbreaks. 


DATA AVAILABILITY STATEMENT 


Publicly available datasets were analyzed in this study. 
This data can be found here: https://ourworldindata.org/ 
coronavirus; https://www.bsg.ox.ac.uk/research/research- 
projects/covid- 19- government-response-tracker. 


AUTHOR CONTRIBUTIONS 


SM and TR contributed equally to conceptualization, 
methodology, data analysis, and writing of the manuscript. 
Both authors have read and agreed to the final version of 
the manuscript. 


CODE AVAILABILITY 


All R code used to produce the reported results are available on 
Github: https://github.com/ruettenauer/Replication-npi-covid. 


ACKNOWLEDGMENTS 


We owe thanks to Thorsten Kneip and Tzvetan Moev for valuable 
comments on earlier drafts of the manuscript. Moreover, we are 
grateful to seminar participants at LMU Munich, Max Planck 
Institute for Social Law and Social Policy, and University of 
Oxford for helpful feedback. 


SUPPLEMENTARY MATERIAL 


The Supplementary Material for this article can be found 
online at: https://www.frontiersin.org/articles/10.3389/fpubh. 
2022.820642/full#supplementary-material 





Frontiers in Public Health | www.frontiersin.org 


April 2022 | Volume 10 | Article 820642 


Mader and Rüttenauer 


NPIs and COVID-19 Mortality 





REFERENCES 


10. 


I 


12. 


13; 


14. 


15. 


16. 


L7; 


18. 


19. 


20. 


. Hale T, Angrist N, Goldszmidt R, Kira B, Petherick A, Phillips T, 


et al. A global panel database of pandemic policies (Oxford COVID- 
19 Government Response Tracker). Nat Hum Behav. (2021) 5:529- 
38. doi: 10.1038/s41562-021-01079-8 


. World Health Organization. Non-Pharmaceutical Public Health Measures for 


Mitigating the Risk and Impact of Epidemic and Pandemic Influenza: Annex: 
Report of Systematic Literature Reviews. (2019). Available online at: https:// 
www.who.int/influenza/publications/public_health_measures/publication/ 
en/ (accessed November 12, 2021). 


. Engzell P, Frey A, Verhagen MD. Learning loss due to school closures 


during the COVID-19 pandemic. Proc Natl Acad Sci USA. (2021) 
118:e2022376118. doi: 10.1073/pnas.2022376118 


. Joffe AR. COVID-19: rethinking the lockdown groupthink. Front Public 


Health. (2021) 9:625778. doi: 10.3389/fpubh.2021.625778 


. Kosfeld R, Mitze T, Rode J, Walde K. The Covid—19 containment effects of 


public health measures A spatial difference-in-differences approach. J Reg Sci. 
(2021) 61:799-825. doi: 10.1101/2020.12.15.20248173 


. Melnick ER, Ioannidis JPA. Should governments continue lockdown to slow 


the spread of covid-19? BMJ. (2020) 369:m1924. doi: 10.1136/bmj.m1924 


. Witteveen D, Velthorst E. Economic hardship and mental health 


complaints during COVID-19. Proc Natl 
117:27277-84. doi: 10.1073/pnas.2009609117 


Acad Sci USA. (2020) 


. Wood SN. Inferring UK COVID-19 fatal infection trajectories from daily 


mortality data: were infections already in decline before the UK lockdowns? 
Biometrics. (2021) 1-14. doi: 10.1111/biom.13462. [Epub ahead of print]. 


. Nuffield Council on Bioethics. Ethical Considerations in Responding to 


the COVID- 19 Pandemic. (2020). Available online at: https://www. 
nuffieldbioethics.org/assets/pdfs/Ethical-considerations-in-responding- 

to- the- COVID- 19- pandemic.pdf (accessed November 12, 2021). 

Auger KA, Shah SS, Richardson T, Hartley D, Hall M, Warniment A, et al. 
Association between statewide school closure and COVID-19 incidence and 
mortality in the US. JAMA. (2020) 324:859-70. doi: 10.1001/jama.2020.14348 
Chernozhukov V, Kasahara H, Schrimpf P. Causal impact of masks, policies, 
behavior on early covid-19 pandemic in the U.S. J Econom. (2021) 220:23- 
62 doi: 10.1016/j.jeconom.2020.09.003 

Figueiredo AM de, Codina AD, Figueiredo D, Saez M, Leon AC. 
Impact of lockdown on COVID-19 incidence and mortality in China: 
an interrupted time series study. Bull World Health Org. (2020) 1- 
19. doi: 10.2471/BLT.20.256701 

Dergiades T, Milas C, Panagiotidis T. Effectiveness of government policies in 
response to the COVID-19 outbreak. SSRN. (2020) doi: 10.2139/ssrn.3602004 
Flaxman S, Mishra S, Gandy A, Unwin HJT, Mellan TA, Coupland H, et al. 
Estimating the effects of non-pharmaceutical interventions on COVID-19 in 
Europe. Nature. (2020) 584:257-61. doi: 10.1038/s41586-020-2405-7 

Leffler CT, Ing E, Lykins JD, Hogan MC, McKeown CA, Grzybowski A. 
As- sociation of country-wide coronavirus mortality with demographics, 
testing, lockdowns, and public wearing of masks. Am J Trop Med Hyg. (2020) 
103:2400-11. doi: 10.4269/ajtmh.20-1015 

Sears J, Villas-Boas JM, Villas-Boas V, Villas-Boas SB. Are we #Stayinghome to 
flatten the curve? medRxiv. (2020). doi: 10.1101/2020.05.23.20111211 

Sharma M, Mindermann S, Rogers-Smith C, Leech G, Snodin B, Ahuja 
J, et al. Understanding the effectiveness of government interventions 
against the resurgence of COVID-19 in Europe. Nat Commun. (2021) 
12:5820. doi: 10.1038/s41467-021-26013-4 

Siedner MJ, Harling G, Reynolds Z, Gilbert RF, Haneuse S, Venkataramani 
AS, et al. Social distancing to slow the US COVID-19 epidemic: 
longitudinal pretest-posttest comparison group study. PLoS Med. (2020) 
17:e1003244. doi: 10.1371/journal.pmed.1003244 

Stokes J, Turner AJ, Anselmi L, Morciano M, Hone T. The 
relative effects of non-pharmaceutical interventions on early Covid- 
19 mortality: natural experiment in 130 countries. medRxiv. 
(2020) doi: 10.1101/2020.10.05.20206888 

Gatto M, Bertuzzo E, Mari L, Miccoli S, Carraro L, Casagrandi R, et al. 
Spread and dynamics of the COVID-19 epidemic in Italy: effects of 
emergency containment measures. Proc Natl Acad Sci USA. (2020) 117:10484- 
91. doi: 10.1073/pnas.2004978117 


21. 


22. 


23. 


24. 


25, 


26. 


28. 


29, 


30. 


3L, 


32. 


33: 


34, 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


Brauner JM, Mindermann S, Sharma M, Johnston D, Salvatier J, Gavenčiak T, 
et al. Inferring the effectiveness of government interventions against COVID- 
19. Science. (2021) 371:eabd9338. doi: 10.1126/science.abd9338 

Xu Y. Generalized synthetic control method: causal inference with interactive 
fixed effects models. Polit Anal. (2017) 25:57-76. doi: 10.1017/pan.2016.2 
Abadie A, Diamond A, Hainmueller J. Synthetic control methods for compar- 
ative case studies: estimating the effect of Californias tobacco control 
program. J Am Stat Assoc. (2010) 105:493-505. doi: 10.1198/jasa.2009.ap08746 
Abadie A, Diamond A, Hainmueller J. Comparative politics and the synthetic 
control method. Am J Polit Sci. (2015) 59:495-510. doi: 10.1111/ajps.12116 
Abadie A. Using synthetic controls: feasibility, data requirements, 
and methodological aspects. J Econ Lit. (2021) 59:391- 
425. doi: 10.1257/jel.20191450 

Goodman-Bacon A, Marcus J. Using Difference-in-Differences to identify 
causal effects of COVID-19 policies. Surv Res Methods. (2020) 14:153- 
8. doi: 10.2139/ssrn.3603970 


. Roser M, Ritchie H, Ortiz-Ospina E, Hasell J. Coronavirus Pandemic (COVID- 


19). (2020). Available online at: https://ourworldindata.org/coronavirus 
(accessed November 12, 2021). 

Wooldridge JM. Econometric Analysis of Cross Section and Panel Data. Cam- 
bridge, MA: MIT Press (2010). 

Goodman-Bacon A. Difference-in-differences with variation in treatment 
timing. J Econom. (2021) 225:254-77. doi: 10.1016/j.jeconom.2021.03.014 
Available online at: https://www.sciencedirect.com/science/article/pii/ 
$0304407621001445 

Callaway B, Sant'Anna PH. Difference-in-differences with multiple time 
periods. J Econom. (2020) 225:200-30. doi: 10.1016/j.jeconom.2020.12.001 
Riittenauer T, Ludwig V. Fixed effects individual slopes: accounting and 
testing for heterogeneous effects in panel data or other multilevel models. 
Sociol Methods Res. (2020) 1-42. doi: 10.1177%2F0049124120926211. [Epub 
ahead of print]. 

Callaway B, Li T. Policy Evaluation During a Pandemic. arXiv. (2021). 
Available online at: http://arxiv.org/pdf/2105.06927v1 (accessed November 
12, 2021). 

Athey S, Bayati M, Doudchenko N, Imbens G, Khosravi K. Matrix comple- 
tion methods for causal panel data models. J Am Stat Assoc. (2021) 59:1716- 
30. doi: 10.1080/01621459.2021.1891924 

Liu L, Wang Y, Xu Y. A Practical Guide to Counterfactual Estimators 
for Causal Inference With Time-Series Data. SSRN 
(2020). doi: 10.2139/ssrn.3555463 

Domenech de Celles M, Casalegno JS, Lina B, Opatowski L. Influenza 
May facilitate the Spread of SARS-CoV-2. (2020). doi: 10.1101/2020.09.07.201 
89779 

Poirier C, LuoW, Majumder MS, Liu D, Mandl KD, Mooring TA, et al. 
The role of environmental factors on transmission rates of the COVID- 
19 outbreak: an initial assessment in two spatial scales. Sci Rep. (2020) 
10:17002. doi: 10.1038/s41598-020-74089-7 

Sajadi MM, Habibzadeh P, Vintzileos A, Shokouhi S, Miralles-Wilhelm 
F, Amoroso A. Temperature, humidity, and latitude analysis to estimate 
potential spread and seasonality of coronavirus disease 2019 (COVID-19). 
JAMA Netw Open. (2020) 3:e2011834. doi: 10.1001/jamanetworkopen.2020. 
11834 

Runkle JD, Sugg MM, Leeper RD, Rao Y, Matthews JL, Rennie 
JJ. Short-term effects of specific humidity and temperature on 
COVID-19 morbidity in select US cities. Sci Total Environ. (2020) 
740:140093. doi: 10.1016/j.scitotenv.2020.140093 

Hersbach H, Bell B, Berrisford P, Hirahara S, Horanyi A, Mufioz-Sabater 
J, et al. The ERA5 global reanalysis. Q J R Meteorol Soc. (2020) 146:1999- 
2049. doi: 10.1002/qj.3803 

Wodtke GT. Regression-based adjustment for time-varying confounders. 
Sociol Methods Res. (2020) 49:906-46. doi: 10.1177/00491241187 
69087 

Wodtke GT, Alaca Z, Zhou X. Regression-with-residuals estimation of 
marginal effects: a method of adjusting for treatment-induced confounders 
that may also be effect modifiers. J R Stat Soc Ser A. (2020) 183:311- 
32. doi: 10.1111/rssa.12497 

Sebhatu A, Wennberg K, Arora-Jonsson S, Lindberg SI. Explaining the 
ho- mogeneous diffusion of COVID-19 nonpharmaceutical interventions 


Cross-Sectional 





Frontiers in Public Health | www.frontiersin.org 


April 2022 | Volume 10 | Article 820642 


Mader and Rüttenauer 


NPIs and COVID-19 Mortality 





43. 


44. 


45. 


46. 


47. 


48. 


across heterogeneous countries. Proc Natl Acad Sci USA. (2020) 117:21201- 
8. doi: 10.1073/pnas.2010625117 

Askitas N, Tatsiramos K, Verheyden B. Estimating worldwide effects 
of non-pharmaceutical interventions on COVID-19 incidence and 
population mobility patterns using a multiple-event study. Sci Rep. (2021) 
11:1972. doi: 10.1038/s41598-021-81442-x 

Charoenwong B, Kwan A, Pursiainen V. Social connections with COVID-19- 
affected areas increase compliance with mobility restrictions. Sci Adv. (2020) 
6:eabc3054. doi: 10.1126/sciadv.abc3054 

Weill JA, Stigler M, Deschenes O, Springborn MR. Social distancing responses 
to COVID-19 emergency declarations strongly differentiated by income. Proc 
Natl Acad Sci USA. (2020) 117:19658-60. doi: 10.1073/pnas.2009412117 
Haug N, Geyrhofer L, Londei A, Dervic E, Desvars-Larrive A, Loreto 
V, et al. Ranking the effectiveness of worldwide COVID-19 government 
interventions. Nat Hum Behav. (2020) 4:1303-12. doi: 10.1038/s41562-020-0 
1009-0 

Lai S, Ruktanonchai NW, Zhou L, Prosper O, Luo W, Floyd JR, 
et al. Effect of non-pharmaceutical interventions to contain COVID- 
19 in China. (2020) 585:410-3. doi: 10.1038/s41586-020- 
2293-x 

Vuong Q-H, et al. Covid-19 vaccines production and societal immunization 
under the serendipity-mindsponge-3D knowledge management 
theory and conceptual framework. Humanit Soc Sci Commun. (2022) 
9:22. doi: 10.1057/s41599-022-01034-6 


Nature. 


49. Mills, MC, Rüttenauer, T. The effect of mandatory COVID-19 certificates 
on vaccine uptake: synthetic-control modeling of six countries. 
Lancet Public Health. (2022) 7:e15-22. doi: 10.1016/S2468-2667(21)0 
0273-5 

50. O'Driscoll M, et al. Age-specific mortality and immunity patterns of SARS- 
CoV-2. Nature. (2020) 590:140-5. doi: 10.1038/s41586-020-2918-0 


Conflict of Interest: The authors declare that the research was conducted in the 
absence of any commercial or financial relationships that could be construed as a 
potential conflict of interest. 


Publisher’s Note: All claims expressed in this article are solely those of the authors 
and do not necessarily represent those of their affiliated organizations, or those of 
the publisher, the editors and the reviewers. Any product that may be evaluated in 
this article, or claim that may be made by its manufacturer, is not guaranteed or 
endorsed by the publisher. 


Copyright © 2022 Mader and Rüttenauer. This is an open-access article distributed 
under the terms of the Creative Commons Attribution License (CC BY). The use, 
distribution or reproduction in other forums is permitted, provided the original 
author(s) and the copyright owner(s) are credited and that the original publication 
in this journal is cited, in accordance with accepted academic practice. No use, 
distribution or reproduction is permitted which does not comply with these terms. 





Frontiers in Public Health | www.frontiersin.org 


April 2022 | Volume 10 | Article 820642 


